Deep Q-Learning for Nash Equilibria: Nash-DQN

Model-free learning for multi-agent stochastic games is an active area of research. Existing reinforcement learning algorithms, however, are often restricted to zero-sum games, and are applicable only in small state-action spaces or other simplified settings. Here, we develop a new data efficient Deep-Q-learning methodology for model-free learning of Nash equilibria for general-sum stochastic games. The algorithm uses a local linear-quadratic expansion of the stochastic game, which leads to analytically solvable optimal actions. The expansion is parametrized by deep neural networks to give it sufficient flexibility to learn the environment without the need to experience all state-action pairs. We study symmetry properties of the algorithm stemming from label-invariant stochastic games and as a proof of concept, apply our algorithm to learning optimal trading strategies in competitive electronic markets.Comment: 16 pages, 4 figure

Economic Effects of Lifting the Spring Load Restriction Policy in Minnesota

Spring load restrictions (SLR) regulate the weight per axle carried by heavy trucks during the spring thaw period. This policy aims to reduce pavement damage caused by heavy vehicles and extend the useful life of roads, but it also imposes costs on the trucking industry due to detouring or increased number of truckloads. Although the policies have been implemented for many years, their resulting economic effect has been unclear. The Minnesota Local Road Research Board (LRRB) and the Minnesota Department of Transportation (Mn/DOT) sponsored a cost/benefit study of spring load restrictions in Minnesota. The study, based on the results of surveys of industry costs, a pavement performance model, and a freight demand model, concludes that the benefits of lifting the existing SLR policy outweigh the additional costs. Roadways operating at 5-tons require additional study; however, current analysis warrants repealing SLR and keeping roadways operating year-round at 9-tons. The cost of additional damage should be recovered from those who benefit from the change in policy.

Deep GrabCut for Object Selection

Most previous bounding-box-based segmentation methods assume the bounding box tightly covers the object of interest. However it is common that a rectangle input could be too large or too small. In this paper, we propose a novel segmentation approach that uses a rectangle as a soft constraint by transforming it into an Euclidean distance map. A convolutional encoder-decoder network is trained end-to-end by concatenating images with these distance maps as inputs and predicting the object masks as outputs. Our approach gets accurate segmentation results given sloppy rectangles while being general for both interactive segmentation and instance segmentation. We show our network extends to curve-based input without retraining. We further apply our network to instance-level semantic segmentation and resolve any overlap using a conditional random field. Experiments on benchmark datasets demonstrate the effectiveness of the proposed approaches.Comment: BMVC 201

A Framework for Analyzing the Effects of Spring Load Restriction

Spring Load Restrictions (SLR) impose load restrictions on heavy trucks during the spring thaw period. Although the policies have been implemented for many years, we are still unsure of their economic effects on truckers. This paper overviews practices around the world and sets up a framework to estimate the Benefit/Cost of the SLR policy. A freight demand model in Minnesota was built to estimate the impacts of SLR on the freight transportation pattern. The model allows various policy scenarios to be tested before being tested in practice. A preliminary result of the freight demand model shows the SLR policy increased truck Vehicle Kilometers of Travel (VKT) in Lyon County, Minnesota by about 13 percent.Spring load restrictions, Benefit/Cost analysis, EMME/2, Freight demand model

Economic Effects of Lifting the Spring Load Restriction Policy in Minnesota

Spring load restrictions (SLR) regulate the weight per axle carried by heavy trucks during the spring thaw period. This policy aims to reduce pavement damage caused by heavy vehicles and extend the useful life of roads, but it also imposes costs on the trucking industry. A cost/benefit study, based on the results of surveys of industry costs, a pavement performance model, and a freight demand model, concludes that the benefits of lifting the existing SLR policy outweigh the additional costs. The cost of additional damage should be recovered from those who benefit from the change in policy